AD-A236  705 


A  Computational  Model  for  the  Stereoscopic 
Optics  of  a  Head-Mounted  Display 


TR91-009 
February,  1991 


Warren  Robinett 
Jannick  P.  Rolland 


3T».Ts:M33rr  a  | 

At5|>73Ve<l  tor  pi-'blic  rai«as3<  | 


The  University  of  North  Carolina  at  Chapel  Hill 
Department  of  Computer  Science 
CB#3175,  Sitterson  Hall 
Chapel  Hill,  NC  27599-3175 


This  research  was  supported  ia  part  by  the  Defease  Advanced  Project  Research 
Agency,  contract  #  DAEA18-90-C-0044  and  also  by  the  Office  of  Naval  Research 
contract  #  N00014-86-K-0680. 

UNC  is  an  Equal  Opportuaity/AfSrmative  Action  Institution. 


91.-01695 

■li 


91  b  10  0  34 


A  Computational  Model  for  the  Stereoscopic  Optics 
of  a  Head-Mounted  Display 


Warren  Robinett 
Jannick  P.  RoUand 

University  of  North  Carolina,  Computer  Science  Department 
Chapel  Hill.  NC  27599-3175,  USA 


ABSTRACT 

For  stereoscopic  photography  or  telepresence,  orthostereoscopy  occurs  when  the  perceived  size,  shape,  and  relative  position  of 
objects  in  ti^  three-dimensional  scene  being  viewed  match  those  of  the  physical  objects  in  front  of  the  camera.  In  Virtual 
Reality,  the  simulated  scene  has  no  physical  counterpart,  so  orthostereoscopy  must  be  defined  in  this  case  as  constancy,  as  the 
head  moves  around,  of  the  perceived  size,  shape  and  relative  positions  of  the  simulated  objects. 

Achieving  this  constancy  requires  that  the  computational  model  used  to  generate  the  graphics  match  the  physical  geometry  of 
the  head-mounted  display  being  used.  This  geometry  includes  the  optics  used  to  image  the  displays  and  the  placement  of  the 
displays  with  respect  to  the  eyes.  The  model  may  fail  to  match  the  geometry  because  model  parameters  are  difficult  to 
measure  accurately,  or  because  the  model  itself  is  in  en<x.  Two  common  modeling  errors  are  ignoring  the  distortion  caused  by 
the  optics  and  ignoring  the  variation  in  interpupillary  distance  across  different  users. 

A  computational  model  for  the  geometry  of  a  head-mounted  display  is  presented,  and  the  parameters  of  this  model  for  the  VPL 
EyePhone  are  calculated. 


1.  INTRODUCTION 


1.1  The  Problem:  Computing  the  Correct  Stereoscopic  Images  in  Virtual  Reality 


As  you  move  through  the  world,  images  of  the  objects  that  surround  you  fall  onto  your  retinas.  As  you  move  past  a  fixed 
object,  seeing  it  from  various  angles,  the  size  and  shape  of  the  images  on  your  retinas  change,  yet  you  effortlessly  and 
unconsciously  perceive  the  object  to  have  a  stable  position,  shape  and  size.  This  innate  perceptual  ability,  honed  by  your 
daily  experience  ever  since  infancy,  is  so  fundamental  and  habitual  that  it  seems  almost  absurd  to  talk  about  objects  which 
could  change  their  position  or  shape  or  size  depending  on  how  you  moved  your  head. 

Yet  the  current  state  of  the  art  in  Virtual  Reality  (VR)  gives  us  simulated  objects  which  change  their  position,  size  and  shape 
as  the  head  moves.  The  location  of  these  objects  appears  to  change  as  the  head  moves  around,  and  their  size  and  shape  appear 
to  change  depending  on  whether  they  are  being  viewed  directly  in  front  of  the  user’s  head  or  off  to  the  side. 


In  Virtual  Reality,  a  Head-Mounted  Display  (HMD)  and  a  head-tracker  are  used  to  rapidly  measure  head  position  and  create  an 
image  for  each  eye  appropriate  to  its  instantaneous  viewpoint  The  HMD-user  can  then  see  simulated  objects  from  different 
points  of  view  as  the  head  moves.  However,  it  is  difficult  to  correctly  calculate  the  images  to  be  painted  onto  the  display 
screens  of  the  HMD.  The  user’s  eyes  and  brain  (and  vestibular  system)  are  very  sensitive  to  inconsistencies. 


The  computational  problem  of  calculating  the  correct  stereoscopic  images  in  VR  -  getting  the  perceived  objects  to  have  the 
right  position,  size  and  shape  -  is  the  same  problem  that  faces  the  designers  of  stereoscopic  photogr^hy  and  telepresence 
systems.  For  these  systems,  orthostereoscopy  occurs  when  the  perceived  size,  shape,  and  relative  position  of  objects  in  the 
three-dimensional  scene  being  viewed  match  those  of  the  physical  objects  in  front  of  the  camera.  In  Virtual  Reality,  the 

simulated  scene  has  no  physical  counterpart,  so  orthostereoscopy  must  be  defined  in  this  case  as  constancy,  as  the  head  moves _ 

around,  of  the  perceived  size,  shape  and  relative  positions  of  the  simulated  objects.  To  calculate  (Mihostereoscopic  images,  the 
display  code  must  precisely  model  the  geometry  of  the  HMD  system  upon  which  the  images  will  be  viewed.  This  includes  — 


the  relative  positions  of  the  display  screens,  the  optics  and  the  eyes.  The  relationship  between  the  screen  and  the  virtual  image 
of  it  must  also  be  modeled. 

This  paper  addresses  only  the  static  image  generation  problem.  To  simulate  objects  that  are  spatially  stable,  temporal 
problems  must  also  be  solved,  but  those  problems  are  outside  the  scope  of  this  paper. 


1.2  Prior  Work 

Since  Ivan  Sutherland  built  the  first  HMD  in  1968,  several  HMD  systems  have  been  built  The  display  code  for  each  system 
defined  an  implicit  model  for  the  particular  geometry  of  each  HMD.  The  authors  of  the  display  code  for  each  system  structured 
the  code  as  they  judged  appropriate,  and  it  is  difficult  to  know  the  precise  details  of  their  display  code  from  what  has  been 
published.  It  appears  that  most  HMD  systems  treated  their  optics  simply  as  a  magnifier,  ignoring  distortion  introduced  by  the 
optics. 

In  Sutherland’s  HMD,  tiny  half-inch  monochrome  QlTs  were  the  display  devices,  and  the  virtual  images  seen  through  the 
optics  subtended  an  angle  of  40  degrees  and  appeared  to  be  at  a  distance  of  18  inches  in  front  of  the  eyes  [SUTH65] 

[SUTH68].  Half-silvered  mirrors  superimposed  the  computer  graphics  onto  the  user’s  direct  view  of  the  real  world.  Later 
versions  of  the  HMD  were  stereoscopic.  The  stereoscopic  HMD  had  both  a  mechanical  adjustment  for  inter-pupillary  distance 
(IPD)  and  a  software  adjustment  for  the  virtual  eye  separation.  This  HMD  system  was  moved  to  the  University  of  Utah,  and 
essentially  the  same  system  was  used  by  several  students  there  [VICK74]  [CLAR76]. 

In  1983,  Mark  Callahan  at  MIT  built  a  see-through  HMD  similar  to  Sutherland’s  [CALL83].  It  used  half-silvered  mirrors 
mounted  on  eyeglass  frames,  2-inch  monochrome  CRTs,  and  a  bicycle  helmet  An  optical  disk  was  used  to  rapidly  display 
pre-recorded  images  in  response  to  head  movements. 

In  1985  at  NASA  Ames  Research  Center,  Mike  McGreevy  and  Jim  Humphries  built  a  non-see-ihrough  HMD  from 
monochrome  LCD  pocket  television  displays,  a  motorcycle  helmet,  and  the  LEEP  wide-angle  stereoscopic  optics  [HOWL83]. 
This  HMD  was  later  improved  by  Scott  Fisher,  Warren  Robinett  and  others  [FISH86].  The  display  code  for  this  system 
treated  the  LEEP  optics  as  a  simple  magnifier.  The  LEEP  optics  system  has  very  large  exit  pupils  and  therefore  no 
mechanical  IPD  adjustment 

At  Wright-Patterson  Air  Force  Base  in  the  Seventies  and  Eighties,  Tom  Furness  directed  a  program  which  developed 
prototype  HMDs  for  use  in  military  aircraft  [GLIN86].  The  system  was  called  Visually-Coupled  Airborne  Systems  Simulator 
(VCASS),  and  several  prototype  see-through  HMDs  with  custom-designed  optics  were  developed  there  [KOCI86]. 

CAE  Electronics  of  Quebec  has  developed  a  fiber-optic  helmet-mounted  display  system  (FOHMD),  intended  for  flight 
simulators  [CAE86]  [HEND89].  Four  light  valves  i^ve  the  HMD  through  fiber-optic  cables  and  pancake  optics  allow  the 
user  to  see-through  to  the  flight  simulator’s  control  panel.  There  is  a  mechanical  adjustment  for  IPD.  The  binocular  field-of- 
view  (FOV)  is  135  degrees  horizontally  by  64  degrees  vertically,  and  it  also  has  a  25  x  19  degree  high  resolution  inset  field. 

Several  prototype  HMDs  have  been  constructed  here  at  the  University  of  North  Carolina  at  Chapel  Hill  [HOLL87] 
(CHUN89].  In  1985,  a  see-through  HMD  was  made  from  color  LCD  displays,  half-silvered  mirrors,  magnifying  lenses,  and  a 
pilot’s  instrument  training  hood.  The  FOV  was  approximately  25  degrees  horizontally.  A  later  model,  built  at  the  Air  Force 
Institute  of  Technology  (AFIT)  with  UNC  collaboration,  was  made  from  color  LCDs,  very  strong  reading  glasses,  and  a 
bicycle  helmet.  Its  FOV  is  about  55  degrees  horizontally  and  it  is  not  a  see-through  HMD.  We  are  currently  designing  a  see- 
through  HMD  for  medical  imaging  applications.  It  will  incorporate  custom-designed  optics. 

In  1988,  VPL  Research  of  Redwood  City,  California,  began  selling  the  EyePhone,  the  first  commercially  available  HMD 
[VPL89].  It  uses  color  LCD  displays  and  the  LEEP  optics.  It  attaches  to  the  head  with  a  rubber  diving  mask  and  fabric  straps. 
It  is  not  see-through. 
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In  1989,  Eric  Howleu,  the  inventor  of  the  LEEP  optics,  put  together  a  commercial  HMD,  the  LEEPvideo  System  I.  It  used 
monochrome  LCD  displays,  the  LEEP  optics,  and  a  head-mounting  apparatus  designed  by  Howleu.  Howleu  subsequently 
introduced  improved  models  which  use  color  LCDs  and  have  a  wider  FOV  [LEEP90]. 

In  1989,  Reflection  Technologies  of  Waltham,  Massachusetts,  produced  a  product  called  Private  Eye,  a  single  eye 
monochrome  HMD  [nM£89].  It  uses  a  vibrating  mirror  and  an  LED  linear  array  to  produce  a  2D  image.  The  horizontal  FOV 
is  about  25  degrees. 


1.3  Remaining  Problems 

Generating  correct  stereoscopic  images  for  an  HMD  is  a  difficult  task.  The  display  code  for  each  HMD  system  embodies  an 
implicit  computational  model  of  the  geometry  of  the  HMD,  and  there  are  many  sources  of  error  which  must  be  compensated 
for.  In  current  practice,  most  of  these  models  are  inadequate  because  they  ignore  certain  sources  of  error.  Also,  since  these 
models  are  embodied  only  in  the  display  code  of  the  HMD  systems,  they  are  difficult  to  comprehend,  and  are  not  accessible  to 
most  people.  It  is  difficult  to  compare  the  computational  models  of  different  HMD  systems. 

The  display  software  often  ignores  the  system  optics.  But  because  the  optics  actually  do  affect  the  images  seen  by  the  eyes, 
some  of  the  parameters  in  the  display  software  are  tweaked  to  get  the  convergence  and  FOV  of  the  HMD  U)  be  roughly 
correct.  This  type  of  measurement  of  the  parameters  of  the  HMD  system  by  subjective  calibration  by  the  users  is  inaccurate 
compared  with  calculating  the  model  parameters  from  the  specifications  of  the  optics  and  display  screens. 

Another  problem  is  that  most  current  HMD  systems  ignore  the  variation  in  IPD  across  different  users.  In  this  case,  wide  and 
narrow-eyed  users  will  have  different  size  perceptions  of  the  same  simulated  objects. 

This  paper  laesents  an  explicit  computational  model  for  generating  orthostereosct^ically  coirect  images.  Implementing 
display  software  which  follows  this  model  will  produce  stereoscopic  images  which  are  orihostereoscopic  --  simulated  objects 
wiU  be  perceived  as  three-dimensional  and  will  be  undistorted  and  correctly  sized. 

We  first  survey  the  various  sources  of  error  which  cause  incorrect  stereoscopic  images  to  be  generated.  We  then  introduce  a 
computational  model  of  the  geometry  of  an  HMD  which  models  the  optics,  the  distance  between  the  user’s  eyes,  and  the 
relative  positions  of  the  eyes,  optics,  and  display  screens.  This  model  allows  correct  CHthostereoscopic  images  to  be 
calculate.  Finally,  we  calculate  the  model  parameters  for  the  VPL  EyePhone. 


2.  SOURCES  OF  ERROR 

There  is  a  very  precise  correlation  between  the  movements  of  one’s  head  and  the  images  of  an  object  that  are  formed  on  the 
retinas  from  moment  to  moment.  This  correlation  can  be  described  by  simple  geometry:  the  object’s  images  are  projected 
onto  the  retinas,  and  the  retinal  images  depend  only  on  the  object’s  shape  and  the  relative  position  of  the  two  eyes  with 
respect  to  the  object  An  HMD  system  attempts  to  mimic  this  geometry,  painting  images  onto  display  screens  in  front  of  the 
eyes  to  fool  the  eyes  and  brain  into  perceiving  three-dimensional  objects.  If  the  wrong  images  are  painted  onto  the  screens,  the 
user  is  not  able  to  perceive  the  simulated  object  correctly.  The  object  will  either  be  distorted,  or  there  will  be  no  perception  of 
a  3D  object  at  all. 

The  wrong  images  are  painted  onto  the  screens  either  because  of  errors  and  inaccuracies  in  the  head-tracking,  or  because  of 
errors  in  the  software  which  controls  the  image  generation.  While  the  tracking  hardware  can  introduces  significant  error,  it  is 
the  display  software  which  is  the  subject  of  this  paper. 

We  will  introduce  a  computational  model  for  a  Head-Mounted  Display,  and  say  that  the  display  software 
implements  this  model.  For  the  display  software  to  generate  the  images  required  to  give  the  HMD-user  the  perception  of 
undistorted  objects,  the  software  must  take  into  account  the  physical  geometry  of  each  hardware  component  that  affects  the 
final  image  seen  by  the  eyes.  This  geometry  includes  the  display  screens,  the  optics  used  to  image  the  displays,  and  the 
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placement  of  the  displays  with  respect  to  the  eyes.  Before  introducing  the  computational  model,  we  discuss  some  common 
errors  in  the  display  code  for  HMDs. 


2.1  Incorrect  convergence 

Both  eyes  are  necessary  for  stereoscopic  vision.  When  the  eyes  are  focussed  on  a  distant  object,  the  lines  of  sight  are  roughly 
parallel,  and  when  focussed  on  a  near  object,  the  lines  of  sight  converge.  The  nearer  the  object,  the  greater  the  convergence. 

A  stereoscopic  HMD  has,  for  each  eye.  a  display  screen  and  an  optical  system  through  which  the  screen  is  viewed.  If  the 
optical  axes  were  parallel  for  the  two  optical  systems  and  if  the  optical  axis  passed  through  the  center  pixel  of  each  screen, 
then  by  illuminating  those  two  center  pixels,  the  user  would  see  a  stereoscopic  image  of  a  point  of  light  at  infinity.  This 
would  be  more  or  less  like  looking  at  a  single  star  in  the  night  sky.  However,  many  HMDs  do  not  satisfy  those  two 
conditions:  turning  on  the  center  pixels  would  either  produce  a  percept  of  a  not-so-distant  point  of  light  in  front  of  the  user,  or 
else  be  too  divergent  to  fuse  at  all  into  a  stereoscopic  percept. 

In  addition  to  the  horizontal  alignment  problem  related  to  convergence  and  divergence,  there  can  also  be  a  vertical 
misalignment  between  the  two  eyes.  This  is  called  dipvergence. 

Creating  stereoscopic  images  with  the  correct  convergence,  when  the  optical  axes  are  not  parallel  or  the  centers  of  the  dispbys 
are  offset  from  the  optical  axes,  requires  corrective  transformations  in  the  computational  model.  Neither  of  these  properties  is 
a  mistake  in  the  design  of  an  HMD,  they  just  make  the  computational  model  a  little  more  complicated.  In  fact,  many  current 
HMDs  have  non-parallel  optical  axes  and  off-center  screens. 


2.2  Accomodation  not  linked  to  convergence 

The  eyes  converge  on  a  nearby  object  by  rotating  inward;  the  lenses  of  the  eyes  simultaneously  accomodate  to  the  distance  of 
the  object  to  bring  it  into  focus.  Thus  convergence  and  accomodation  are  normally  linked.  However,  in  an  HMD  system,  each 
eye  sees  the  virtual  image  of  a  display  screen.  With  respect  to  focus,  the  entire  virtual  image  appears  at  a  fixed  distance  from 
the  eye.  (In  a  physical  scene,  different  parts  of  the  scene  will  be  in  focus  at  different  accomodation  depths.)  Hence,  the  HMD 
user  must  learn  to  decouple  accomodation  and  convergence.  This  problem  cannot  be  overcome  with  any  currently-used  display 
device.  Until  a  display  device  with  variable  focus  is  developed,  this  problem  must  be  accepted  as  a  limitation  of  HMDs. 


2.3  Incorrect  field-of-view 

The  display  for  a  single  eye  in  an  HMD  has  an  FOV,  which  is  the  angle  subtended  by  the  virtual  image  of  the  display  screen, 
as  seen  by  the  eye.  There  is  a  horizontal  and  a  vertical  FOV  corresponding  to  the  left-to-right  and  top-to-bottom  angular 
sweep  across  the  virtual  image  of  the  display.  Let’s  call  these  angles  the  physical  FOV.  To  be  accurate  iii  our  definition  of 
the  FOV,  we  shall  consider  that  a  point  in  object  space  contributes  to  the  physical  FOV  if  the  chief  ray  defined  as  the  ray 
passing  through  that  point  and  the  center  of  the  exit  pupil  of  the  viewing  system  (here  the  pupil  of  the  eye)  is  not  obstructed. 
We  are  somewhat  conservative  therefore  in  our  definition  of  the  FOV  since  a  point  could  be  said  to  belong  to  the  FOV  as 
long  as  at  least  one  ray  reaches  the  image  plane.  This  FOV  is  usually  referred  to  in  the  optics  literature  as  the  total  FOV. 

An  FOV  is  also  specified  in  the  display  code.  This  computational  FOV  determines  how  far  away  the  center  of  projection  is 
from  the  screen  rectangle  in  the  perspective  transformation  that  is  used  to  project  the  3D  virtual  world  onto  the  2D  screen. 
There  is  both  a  horizontal  and  vertical  FOV  in  this  case,  also.  These  are  specified  in  many  graphics  systems  by  giving  the 
aspect  ratio,  the  ratio  of  horizontal  to  vertical  FOV  angles,  and  then  giving  the  vertical  angle. 

Unfortunately,  the  computational  FOV  angles  may  not  match  the  physical  FOV  angles.  As  with  convergence,  the  software 
designer  may  not  know  the  optical  specifications  and  may  be  forced  to  measure  the  FOV  empirically.  To  get  it  right,  the 
position  of  the  center  of  projection  with  respect  to  the  screen  should  be  exactly  at  the  entrance  pupil  of  the  eye. 
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2.4  Failure  to  use  off-center  projection  when  required 

If  the  display  screen  is  perpendicular  to  the  optical  axis  and  off-center  from  the  axis,  then  the  eye  is  off-center  with  respect  to 
the  screen,  and  the  computational  center  of  projection  should  be  off-center,  too.  This  situation  requires  an  off^-center 
perspective  projection,  in  which  the  left,  right,  top  and  bottom  edges  of  the  screen  rectangle  are  specified  independently  of  one 
another,  rather  than  having  left-right  and  top-bottom  be  symmetrical  as  usual.  An  off-center  perspective  projection 
transformation  can  be  set  up  with  the  stand^d  computer  graphics  hardware  using  the  standard  4x4  homogeneous  matrix 
representation  of  the  transformation.  Many  people  even  in  the  computer  graphics  profession  are  unfamiliar  with  the  off-center 
perspective  projection  -  it  is  never  needed  in  normal  computer  graphics  because,  since  the  physical  eye  position  is  unknown, 
a  convenient  one  directly  in  front  of  the  screen  may  as  well  be  assumed. 

Another  mistake  is  to  simply  rotate  the  whole  scene  left  or  right  to  produce  images  for  the  two  eyes.  In  general,  the 
computational  model  needs  to  include  transformations  that  take  into  account  the  position  and  orientation  of  the  screens  with 
respect  to  the  user’s  eyes.  This  is  likely  to  be  a  combination  of  uanslation,  rotation  and  off-center  perspective  projection,  and 
very  unlikey  to  be  just  a  pure  rotation  [S  AUN68]. 


2.5  Inter-pupillary  distance  ignored 

Among  male  and  female  adults,  there  is  a  fairly  wide  variation  in  the  distance  between  the  eyes,  called  the  inter-pupillary 
distance  (IPD).  The  range  is  roughly  53  to  73  mm,  with  the  average  IPD  being  about  63  mm.  Children  have  even  smaller 
IPDs. 

The  variation  in  IPE)  imposes  a  requirement  on  the  HMD  optics  and  display  hardware.  Either  the  exit  pupil  of  the  optics  must 
be  large  enough  to  be  used  by  the  widest-eyed  and  the  narrowest-eyed  people,  or  else  there  must  be  a  mechanical  adjustment 
such  as  is  found  on  binoculars.  Both  of  these  approaches  have  been  used.  For  example,  the  LEEP  optics,  used  in  many 
current  HMDs,  has  a  very  large  exit  pupil.  The  CAE  fiber-optic  HMD  used  for  flight  simulation  requires  a  mechanical 
adjustment  for  each  new  user. 

The  distance  between  the  eyes  is  the  baseline  from  which  a  person  judges  the  distance  to  objects  in  the  physical  world.  The 
convergence  of  the  lines  of  sight  from  the  eyes  to  a  point  of  fixation  can  be  measured  as  an  angle.  As  Figure  2.1  shows,  a 
narrow-eyed  and  a  wide-eyed  person  will  have  different  convergence  angles  when  looking  at  an  object  at  the  same  distance, 
say,  half  a  meter  away. 


Figure  2.1  Eyes  with  narrow  versus  wide  IPDs  looking  at  an  object  from  the  same  distance 


These  two  people  have  different  convergence  angles  yet  both  perceive  the  object  to  be  half  a  meter  away.  Each  person  is 
calibrated  to  his  or  her  own  IPD. 

With  the  mechanical  IPD  adjustment,  the  images  get  piped  into  the  user’s  eyes  and  his  or  her  physical  IPD  has  no  effect  on 
the  images  seen.  With  no  mechanical  adjustment  but  a  large  exit  pupil,  if  the  virtual  images  are  at  optical  infinity,  then  a 
lateral  change  in  eye  position  has  no  effect  on  the  angles  at  which  an  object  appears  -  in  other  words,  in  this  case,  too,  the 
user  gets  the  same  one-size-fits-all  images  regardless  of  the  physical  distance  between  the  eyes.  If  the  virtual  images  are  not  at 
optical  infinity,  the  situation  is  more  complicated,  and  people  of  varying  EPD  are  still  not  going  to  see  images  matched  to 
their  own  IPDs. 
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The  solution  to  this  problem  is  to  measure  the  IPD  for  each  user,  and  have  the  IPD  as  a  user-speciHc  constant  in  the 
computational  model.  If  this  is  done  correctly,  then  each  user  can  see  a  simulated  object  half  a  meter  away  with  a  convergence 
matched  to  his  or  her  own  IPD. 


2.6  Optical  distortion  ignored 

The  display  screens  in  an  HMD  are  too  close  to  the  eyes  to  focus  on  directly,  so  an  optical  system  is  interposed  between  the 
eye  and  the  screen.  The  main  purpose  of  the  optics  is  to  provide  an  image  to  the  user  at  a  comfortable  distance  of 
accomodati'^n  and  with  as  large  a  magnification  as  possible  without  altering  the  image.  The  eye,  looking  into  the  optics,  sees 
a  virtual  image  of  the  display  screen.  The  virtual  image  is  distant  enough  from  the  eye  to  focus  on  easily,  and  large  enough  to 
cover  a  large  swath  of  the  user’s  FOV.  But  the  optics  also  distort  the  image  non-linearly,  causing  lines  that  were  straight  on 
the  display  screen  to  appear  as  curved  in  the  virtual  image. 

Optical  aberrations  are  defects  of  the  image.  They  may  be  described  in  terms  of  the  amount  by  which  a  geometrically-traced 
ray  misses  a  specified  location  in  the  image  plane  formed  by  the  optical  system.  The  displacement  of  the  ray  is  referred  to  as 
the  transverse  ray  aberration.  Most  often,  the  specified  location  for  a  ray  in  the  image  plane  is  that  inferred  from  first-order 
laws  of  image  formation  IHECH74]  [LONG73].  Rays  which  do  propagate  not  only  near  the  optical  axis  but  also  at  shallow 
angles  with  respect  to  the  optical  axis  are  known  as  paraxial  rays.  Under  the  paraxial  approximation,  the  formation  of  images 
is  referred  to  as  first-order,  paraxial,  or  Gaussian  optics.  The  most  common  aberrations  are  spherical  aberration  (SA),  coma, 
astigmatism  (AST),  field  curvature  (FC),  distortion,  and  chromatic  aberrations.  It  should  be  noted  that,  while  SA,  coma, 
AST,  FC,  and  chromatic  aberrations  all  affect  the  sharpness  of  the  image  points  being  formed,  distortion  distinguishes  itself 
from  the  others  since  it  causes  the  image  points  to  be  displaced  transversally  in  a  non-linear  fashion  across  the  FOV  but  does 
not  alter  the  sharpness  of  the  image. 

Transversal  aberrations  can  be  expressed  mathematically  as  a  polynomial  expansion  of  third-order  and  higher-order  terms  in 
both  the  image  height  and  the  height  of  strike  of  the  ray  on  a  reference  sphere  centered  on  the  ideal  image  point  and  passing 
through  the  exit  pupil  of  the  optical  system.  The  ideal  image  point  is  ofien  chosen  to  be  the  paraxial  image  point  The  sum 
of  the  exponents  of  the  aperture  and  field  terms  indicates  the  order  of  the  aberration  represented  by  that  term.  Depending  on 
how  open  the  optical  system  is  and  how  large  the  angles  of  incidence  of  the  rays  on  the  different  surfaces  of  the  optical 
elements  are,  a  system  is  best  described  by  a  third-order  or  a  higher-order  approximation.  The  complexity  of  the  optics  used  is 
usually  such  that  the  sharpness  of  the  images  formed  through  the  optical  system  is  good  enough  for  the  display  resolution 
available  on  the  market  today.  The  distortion  of  the  images,  however,  is  often  disturbing  if  it  has  not  been  corrected  for 
optically.  The  first-order  polynomial  is  linear  and  describes  an  ideal  magnifier  with  no  distortion,  and  since  there  are  no  even 
terms  appearing  in  the  expansion,  the  third-order  polynomial  is  the  simplest  model  of  distortion. 

Non-linear  field  distortion  causes  straight  lines  on  the  screen  to  appear  curved.  This  can  be  corrected  for  in  the  graphics  system 
by  predistorting  the  images  written  onto  the  screens.  A  straight  line  in  the  virtual  image  would  be  created  by  writing  a  curved 
line  onto  the  screen,  such  that  its  curvature  exactly  balanced  out  the  optical  distortion.  This  would  require  that  the  inverse  of 
the  screen-to-virtual-image  distortion  function  be  stored  in  the  graphics  system,  and  that  each  pixel  be  remapped  to  a  new 
location  on  the  screen.  This  is  computationally  expensive. 

Most  current  HMD  systems  just  ignore  the  optical  distortion,  because  they  may  not  have  access  to  the  distortion  function  for 
the  optics,  and  because  of  the  performance  penalty  even  if  they  did  do  the  correction.  This  is  not  an  unreasonable  choice  in  the 
early  stages  of  development  of  an  HMD,  because  the  system  is  usable  even  with  the  optical  distortion. 

However,  one  side-effect  of  non-linear  field  distortion  is  that  there  is  no  single  correct  value  for  the  FOV.  Non-linear 
distortion  causes  the  magnification  to  vary  across  the  FOV.  If  the  computational  FOV  is  set  to  match  the  physical  FOV,  then 
objects  in  the  center  of  the  field  will  appear  to  be  the  wrong  size.  But  if  the  computational  FOV  is  set  to  make  small  central 
objects  appear  to  be  the  right  size,  the  objects  in  the  peripheral  field  will  be  positioned  wrong.  The  only  way  to  avoid  this 
unpleasant  choice  is  to  predistort  the  image  to  correct  the  optical  distortion. 
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If  the  optics  are  designed  specially  for  an  HMD,  then  specific  types  of  aberrations  can  be  minimized.  But  if  optics  designed  for 
another  purpose  are  used,  ^en  you  take  what  you  can  get.  The  LEEP  optics,  used  in  many  current  HMD  systems,  were 
designed  for  stereoscopic  photography,  and  purposely  incorporate  substantial  field  distortion  and  chromatic  aberrations. 


2.7  Transforming  only  polygon  vertices  in  the  presence  of  non-linearities 

Computer  graphics  has  traditionally  gained  much  efficiency  by  representing  graphics  primitives  such  as  lines  and  polygons  as 
collections  of  points,  running  only  the  points  through  the  transformation  pipeline,  and  then  drawing  the  lines  or  polygons 
between  the  transformed  points.  This  works  only  if  all  the  transformations  in  the  pipeline  are  linear.  It  is  tempting  to  run 
only  the  polygon  vertices  through  the  predistortion  function  and  let  the  very  efficient  hardware  fill  in  the  polygons.  But  then 
only  the  vertices  would  be  in  the  right  place  --  the  polygon  edges  would  still  be  curved  by  the  optical  distortion.  Figure  2.2a 
shows  a  simple  case  of  this.  Edges  of  polygons  that  cross  a  large  fraction  of  the  screen  would  be  most  noticeably  curved. 

Another  problem  with  this  approach  is  that  continuity  would  be  lost.  A  vertex  that  touched  an  edge  before  predistortion  and 
scan-conversion  would  not  be  guarranteed  to  do  so  in  the  final  virtual  image.  Gaps  and  holes  would  open  up  everywhere. 
Figure  2.2b  shows  this. 

desired  image  pre-distorted  endpoints  virtual  image  seen  by  eye 


Figure  2.2  Problems  with  predistorting  only  vertices 


3.  OPTICS  MODEL  FOR  A  HEAD-MOUNTED  DISPLAY 

The  purpose  of  the  optics  model  is  to  specify  the  computation  necessary  to  create  orthostereoscopically  correct  images  for  an 
HMD  and  indicate  the  parameters  of  that  system  which  need  to  be  measured  and  incorporated  into  the  model. 


3.1  Single-eye  optics  model 

To  achieve  orthostereoscopy,  the  non-linear  optical  distortion  must  be  corrected  by  remapping  all  the  pixels  on  the  screen 
with  a  predistortion  function.  Linear  graphics  primitives  such  as  lines  and  polygons  are  written  into  a  Virtual  Screen  image 
buffer,  and  then  all  the  pixels  are  shifted  according  to  the  pre-distortion  function  and  wriuen  to  the  Screen  image  buffer  for 
display.  The  predistortion  function  is  the  inverse  of  the  field  distortion  function  for  the  optics,  so  that  the  virtual  image  seen 
by  the  eye  matches  the  image  in  the  Virtual  Screen  buffer.  A  straight  line  in  the  Virtual  Image  buffer  is  predistorted  into  a 
curved  line  on  the  display  screen,  which  is  distorted  by  the  optics  into  a  line  which  is  seen  as  straight. 
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Figure  3.1  shows  the  optics  model  for  a  single  eye. 


Figure  3.1  Single-eye  optics  model 


The  mathematical  representation  of  the  optical  distortion  will  depend  on  the  nature  of  the  optical  system.  For  optics  such  as 
LEEP,  a  third-order  polynomial  approximation  is  adequate.  We  want  to  relate  the  radial  position  rg  of  a  pixel  on  the  screen  to 
the  radial  position  ry  of  the  virtual  image  of  that  pixel.  These  two  quantities  are  measured  in  mm  with  respect  to  the  optical 
axis.  DivWng  rj  and  ry  by  the  object  field  width  wj  and  the  image  field  width  wy,  respectively,  we  get  the  normalized 
position  of  the  pixel  on  the  screen 


fsn  ®  fs  /  ws 


and  the  normalized  position  of  the  virtual  image  of  the  pixel 


fvn  =  ry  /  Wy  , 


(The  paraxial  magnification  of  the  system  is  thus  msy  =  Wy  /  Ws .) 

The  distortion  is  modeled  with  a  third-order  polynomial  approximation 

-I 

Tvn  =  fsn  +  kys  rsn’’ 

in  which  the  coefficient  kys  describes  the  amount  of  distortion  present.  This  can  be  rearranged  algebraically  to 


fvn  =  (1  +  kys  rsn^)  tsn 
and  then  expanded  to  rectangular  coordinates  using 

rsn^  =  xsn^  +  ysn^ 

2  2  2 
rvn^  =  xyn-^  +  yyn^ 

to  give  the  position  of  the  virtual  image  of  the  pixel 

(xvn.  Yvn)  =  (  (1  +  kvs(xsn^  +  Ysn^))  ^sn.  (i  +  kvs(xsn^  +  Ysn^))  Ysn  ) 

from  the  position  (xsn.  ysn)  of  the  pixel  on  the  screen.  These  positions  are  measured  in  the  screen  plane  and  virtual  image 
plane,  respectively,  with  respect  to  the  optical  axis. 
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The  distance  from  the  eye  to  the  virtual  image  plane  Zy  is  a  constant  By  combining  the  above  equations,  we  have  a  function 
that  gives  the  three-dimensional  position  of  the  virtual  image  of  a  pixel  on  the  screen  (xv,yv,Zv)  in  terms  of  its  screen 
coordinates  (xs.ys)  and  some  constants.  We  will  ignore  Zy  from  here  on  since  it  is  constant. 

The  expressions  for  Xyn  and  yyn  can  be  thought  of  as  single-valued  functions  of  two  variables.  If  we  name  the  distortion 
function  D,  then 

(xvn.  yvn)  =  D(  Xsn,  ysn  ) 

and  what  we  need  to  predistort  the  image  on  the  screen  is  the  inverse  D‘^.  There  are  various  ways  this  inverse  function  could 
be  represented  in  the  computer.  An  exact  closed-form  expression  is  not  feasible,  but  a  polynomial  approximation  of  the 
inverse  is  possible. 


fsn  =  fvn  +  ksv  tvn'’ 

(^sn.  ysn)  =  D'kxvn.yvn) 

(Jtsn.  ysn)  =  (  (1  +  l^svf^vn^  +  yvn^))  Xvn.  (1  +  ^svC^vn^  +  yvn^))  yvn  ) 

Note  that  these  two  functions  D  and  D'^  are  each  third-order  polynomial  approximations  and  are  not  exact  inverses  of  one 
another.  The  coefficients  ksv  and  kys  will  be  opposite  in  sign. 

Another  possibility  for  representing  D*^  on  the  computer  is  a  two-dimensional  table  lookup  for  each  of  the  output  variables 
Xsn  and  ysn-  Using  this  approach,  limits  on  table  size  would  probably  make  it  necessary  to  interpolate  between  table  entries. 

Distortion  causes  non-uniform  magnification  across  the  field,  which  causes  the  brightness  also  to  vary  across  the  field.  This 
could  be  compensated  for  on  a  pixel-by-pixel  basis  with  a  brightness  correction  function  B(xs,ys),  but  limitations  of  space 
prevent  us  from  going  into  this  here. 


3.2  Stereoscopic  optics  model 

Figure  3.2  shows  the  stereoscopic  optics  model.  One  pixel  is  illuminated  on  each  screen  (points  A1  and  A2)  and  a  line-of- 
sight  is  drawn  from  each  eye  to  the  virtual  image  of  its  corresponding  pixel  (points  B1  and  B2).  These  two  lines-of-sight 
intersect  at  the  three-dimensional  point  perceived  by  the  user  (point  C).  The  IPD  is  the  baseline  from  which  the  user  makes 
distance  judgements  based  on  the  convergence  angles  to  perceived  points. 

If  the  specifications  for  the  optics  are  known,  and  the  relative  positions  of  the  display  screens,  the  optics,  and  the  eyes  are  also 
known,  then  it  is  possible  to  accurately  calculate  several  important  parameters  ne^ed  in  the  computational  model.  The 
horizontal  and  vertical  FOVs  can  be  calculated  by  starting  from  the  known  positions  of  the  left,  right,  top  and  bottom  edges 
of  the  screen  with  respect  to  the  optics  and  tracing  the  rays  through  the  optics  system  back  to  the  eye. 

The  screen  centers  may  be  offset  from  the  optical  axes  by  a  certain  distance.  The  two  optical  axes  may  be  rotated  with  respect 
to  one  another.  The  screen  center  offsets  and  axis  divergence  angles  can  be  used  to  set  the  convergence  properly  with  no  need 
for  subjective  calibrations.  From  the  known  position  and  orientation  of  the  virtual  image  of  the  screen  relative  to  the  eye,  the 
Eye-To- Virtual-Image  transformation  can  be  calculated  to  be  the  con-ect  mix  of  translation,  rotation,  and  off-center  perspective 
projection.  Translation  is  needed  because  the  eyes  are  in  different  positions  in  space;  rotation  is  needed  if  the  optical  axes  are 
not  parallel;  and  the  projection  is  off-center  if  the  screens  are  off-center  from  the  optical  axes. 
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Figure  3.2  Stereoscopic  optics  model 


Table  3.1  shows  the  sequence  of  starting  from  the  specifications  of  the  optics  and  displays,  measuring  certain  parameters  of 
the  HMD,  and  then  calculating  other  parameters  needed  by  the  computational  model.  The  left  and  right  halves  of  the  optical 
system  are  assumed  to  be  symmetric^.  The  calculations  are  done  for  the  right  side,  so  the  left  edge  of  the  screen  is  on  the 
inside  beside  the  nose  and  the  right  edge  is  on  the  outside. 

Except  for  the  IPD  which  varies  among  users,  every  other  parameter  necessary  to  specify  the  Head-To-Eye,  Eye-To-Virtual- 
Image  and  Virtual-Image-To-Screen  transformations  for  the  left  and  right  eyes  can  be  derived  from  the  specifications  of  the 
optics  and  the  relative  positions  of  the  eyes,  optics  and  screens.  Calculating  these  parameters  is  much  more  accurate  than 
relying  on  subjective  calibration  procedures  in  which  the  parameters  are  adjusted  until  the  image  looks  right.  However,  the 
subjective  measurements  provide  a  nice  check  against  mistakes  in  the  model,  the  measurements  or  the  calculations. 

We  have  defined  a  computational  model  for  the  graphics  computation  without  reference  to  a  particular  HMD  system.  Having 
done  that,  we  now  turn  to  calculating  the  model  parameters  for  a  specific  HMD,  the  VPL  EyePhone. 
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Parameter 


screen  resolution 


anele  between  optical  axes 


distance  between  optical  axes  (at  front 
surface  of  optics) 


eve  relief 


maximum  field-of-view 


object  plane  distance 


distance  from  eve  to  virtual  imaee  plane 


transversal  maenification 


coefficient  of  optical  distortion 


maximum  object  field  radius 


maximum  virtual  image  field  radius 


’  distance  of  user 


screen  center  offset  from  optical  axis 


sition  of  left  screen  edge 


sition  of  right  screen  edge 


ition  of  top  screen  edge 


sition  of  bottom  screen  edge 


object  height  (of  point  on  screen) 


normalized  object  height 


normalized  image  height 
(3rd  order  approximation  of  D 


image  height  (of  point  in  virtual  image 


angular  position  of  point  in  virtual 
image 


angular  position  of  left  edge 
(inner  edge  of  screen) 


angular  position  of  right  edge  (outer 
edge  of  screen) 


angular  position  of  top  edge 


angular  position  of  bottom  edge 


single  eve  vertical  field-of-view 


single  eye  horizontal  ficld-of-view 


overlapped  field-of-view 


binocular  field-of-view 


translation  part  of  viewing 
transformation 


rotation  part  of  viewing  transformation 


perspective  projection 


Where  It  Comes  From 


from  displa 


from  optics  s 


from  optics  spec  or  measure 


or  measure 


ws 


Wv 


IPD 


(Cxc.Cyc) 


(LxsjLvs) 


(Rxs.Rys) 


(Txs.Tys) 


(Bxs,Bys) 


measure 


calculate  from  der  and  optics  spec 


measure 


calculate  from  dpb  and  optics  spec 


calculate  from  dnb  and  optics  spec 


from  optics  spec 


Wv  =  mvs  ws 


measure  user 


measure 


measure 


measure 


measure 


measure 


H  =  (xs  +  ys)'^ 


fsn  =  fs  /  ws 


•“vn  =  Tsn  +  i^vs^sn^ 


rv  =  rvn  wy 


0  =  tan'Htv  /  zv  ) 


from  formula  for  0 


from  formula  for  0 


0T  I  from  formula  for  0 


0B  from  formula  for  0 


FOVy  I  FOVy  =  0T  +  0B 


FOVh  FOVh  =  0L  +  0R 


FOVqv  I  FOVqv  =  2  0l.  0axes 


FQ^bin  I  POV hjn  =  2  0R  -i-  0axes 


Mtrans  (+1PD/2,  0,  0) 


0axes  /2)  around  Y-axis 


Mperspec  use  FOVy,  FOVh,  and  offset 

(  ±Cxs  /  (Rxs  -  Lxs).  Cys/(Tys  -  Bys) ) 


Table  3.1  Calculating  parameters  in  the  optics  model 
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4.  CALCULATING  THE  MODEL  PARAMETERS  FOR  THE  VPL  EYEPHONE 


4.1  Description  of  EyePhone  components 

We  have  used  the  optical  specifications  of  the  LEEP  optics  and  size  and  positioning  of  the  LCD  screens  inside  the  EyePhone 
to  calculate  the  model  parameters  for  the  VPL  EyePhone,  Model  1  (VPL89].  Model  2  of  the  EyePhone  has  identical  optics, 
LCD  displays,  and  positioning  of  the  parts. 

The  LEEP  optics  [HOWL83]  are  wide-angle,  stereoscopic  viewing  tenses.  It  consists  of  a  three-lens  system  in  front  of  each 
eye,  encased  in  a  molded  plastic  mount,  with  a  cut-out  for  the  nose.  It  was  designed  for  a  single  transparency  in  the  object 
plane,  upon  which  arc  two  side-by-side  stereoscopic  photographs  of  a  scene,  each  one  a  square  approximately  64  mm  on  a 
side.  For  an  eye  relief  distance  of  29.4  mm,  the  FOVs  for  each  eye  are  approximately  +45  to  -45  degrees  horizontally,  and 
-HtS  to  -45  degrees  vertically.  The  distance  from  the  center  of  the  rearmost  leas  surface  to  the  object  plane  is  approximately  16 
mm.  The  optical  axes  for  the  two  eyes  are  parallel  and  are  64  mm  apart  The  two  optical  systems  are  bilaterally  symmetrical 
with  respect  to  each  other,  and  each  optical  system  is  radially  symmetrical  around  the  optical  axis,  except  for  the  cut-outs  for 
the  user's  nose  in  the  front  lenses. 

The  construction  of  the  EyePhone  is  rigid  enough  to  keep  the  eyes,  optics,  and  display  screens  in  a  fixed  relationship  to  one 
another,  so  it  is  possible  to  make  accurate  measurements  of  these  relative  positions 

The  EyePhone  uses  the  LEEP  optics  with  two  color  LCD  display  screens  positioned  in  the  LEEP  object  plane.  The  two 
displays  are  positioned  symmetrically  with  respect  to  the  left  and  right  eye  optical  axes.  Figure  4.1  is  a  diagram  of  the 
positions  of  the  LCD  screens  in  the  object  plane. 


64  mm 


Figure  4.1  Position  of  the  EyePhone’s  LCD  displays  in  the  LEEP  object  plane 


Six  important  points  are  labeled  in  Figure  4.1.  Only  one  side  need  be  analyzed  since  the  LCDs  and  optics  are  symmetrical. 


4.2  Calculation  of  EyePhone  field-of-view 

We  use  two  different  methods  to  compute  the  FOVs  for  the  EyePhone.  First,  we  use  the  optics  model  with  the  parameters 
specific  to  the  EyePhone  to  calculate  the  angles  at  which  the  points  in  Figure  4.1  are  seen.  Then,  for  comparison  and 
validation,  we  compute  these  same  angles  by  tracing  rays  through  the  LEEP  optics.  We  do  the  ray-tracing  with  Code  V,  a 
commercial  optical  analysis  software  package  [ORA90].  To  do  the  ray-tracing,  we  use  the  detailed  optical  specifications  of  the 
LEEP  optics,  including  the  position,  curvature,  and  index  of  refraction  for  each  lens  surface  in  the  optical  path. 
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First,  wc  use  the  optics  model.  The  dimensions  given  in  Figure  4.1  are  sufTiccnt  to  determine  the  coordinates  of  ihe  points  A, 
L,  R.  T  and  B  in  the  LEEP  object  plane  (with  respect  to  the  optical  axis  A).  From  these  coordinates,  the  value  of  rg  for  each 
point  can  be  calculated.  We  feed  the  known  posiuons  of  the  edges  of  the  LCD  screens  into  the  optics  model  to  predict  the 
positions  of  the  screen  edges  in  the  virtual  image,  and  from  these  positions  calculate  the  FOVs  for  the  EyePhone.  Wc  shall 
see  that  the  chief  rays  corresponding  to  some  of  the  object  points  on  the  LCDs  are  obstructed  for  der  =  29.4  mm.  In  this 
case  the  FOV  covers  only  part  of  the  LCDs.  The  unseen  part  of  the  LCD  is  said  to  be  vignetted.  The  model  parameters  for 
the  LEEP  optics  as  used  in  the  EyePhone  are 

der  =  29.4  mm 
ws  s  28.1  mm 
Wv  =  271.5  mm 
zv  =  398.2  mm 
kvs  =  0.32 

and  the  equations  from  the  optics  model 


rsn  =  fs  /  Ws 
fvn  *  fsn  +  kvs  fsn"^ 
rv  =  rvn  Wy 
0  =  tan*^(  rv  /  zy  ) 

can  be  used  to  calculate  the  angle  0  for  each  point,  as  shown  in  Table  4.1. 

The  second  method  of  computing  the  angles  is  ray-tracing.  Figure  4.2  "  .  vs  a  horizontal  cross-section  of  the  right-eye  LEEP 
optical  system,  with  the  rays  from  the  points  L,  A  and  R  traced  back  to  the  eye.  The  ray-tracing  was  done  for  each  of  the 
points  A,  LJl,  T  and  B  in  Figure  4.1. 


Figure  4.2  Tracing  rays  from  the  EyePhone’s  LCD  screen  through  the  LEEP  optics 


Table  4.1  shows  the  angular  positions  of  the  virtual  images  of  the  edges  of  the  LCD  screen  as  predicted  by  the  optics  naodel, 
and  as  predicted  by  ray-tracing.  The  comparison  of  the  angles  calculated  by  the  two  methods  shows  that  the  third-order 
approximation  used  in  the  optics  model  is  adequate  for  the  LEEP  optics. 
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0 
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0 

28.1 
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1.000 

1.32 

358.5 

42.0 

45.0 

L 

left  edge  of  screen 

-20.9 

0 

20.9 

0.744 

0.876 

237.9 

30.9 

T 

top  edge  of  screen 

0 

21.8 

21.8 

0.776 

0.926 

251.5 

32.3 

31.8 

B 

bottom  edge  of  screen 

0 

-18.5 

18.5 

0.658 

0.749 

203.4 

27.1 

26.6 

C 

center  of  screen 

6.2 

1.65 

■■■ 

■■1 

■■ 

Table  4.1  EyePhone  FOV  calculation,  assuming  dej-  =  29.4  mm 


From  these  calculations,  we  can  see  that,  for  a  single  eye,  the  horizontal  FOV  is  75.3  degrees  (45.0  +  30.3)  and  the  vertical 
FOV  is  58.4  degrees  (31.8  +  26.6).  These  are  the  physical  FOVs  for  the  EyePhone  --  the  physical  angles  at  which  the  virtual 
images  of  the  edges  of  the  LCD  screen  are  seen  by  the  eye.  To  make  the  graphics  calculation  match  the  physical  FOV,  these 
angles  must  be  incorporated  into  the  calculation.  Here  at  UNC,  we  are  using  the  Pixel-Planes  graphics  engine  [EYLE88]  to 
generate  images  for  the  EyePhone.  Like  many  graphics  systems,  the  Pixel-Planes  graphics  software  accepts  an  angle  for  the 
vertical  FOV,  here  58.4  degrees,  and  an  aspect  ratio,  here  1.289  . 

The  graphics  calculation  must  also  take  into  account  the  fact  that  in  the  EyePhone,  the  center  of  the  LCD  screen  (point  C)  is 
off  the  optical  axis  (point  A).  How  this  off-center  perspective  projection  is  specified  to  the  graphics  software  varies  somewhat 
among  graphics  systems.  For  the  Pixel-Planes  graphics  software,  off-center  projection  is  specified  as  a  horizontal  and  vertical 
offset  in  pixels  from  the  screen  center.  For  the  horizontal  offset,  512  pixels  across  a  54.2  mm  screen  gives  0.106  mm/pixel, 
so  the  6.2  mm  horizontal  offset  is  58.5  pixels.  For  the  vertical  offset,  512  pixels  across  a  40.3  mm  screen  height  gives  0.079 
mm/pixel,  so  the  1.65  mm  vertical  offset  is  21.0  pixels. 

These  calculated  values  for  FOV  and  screen-center  offset  together  with  the  IPD  of  the  user  is  enough  to  specify  precisely  the 
perspective  projection  for  the  two  eyes.  Using  these  projections,  the  convergence  and  FOV  are  guarranteed  to  be  correct,  with 
no  need  for  adjustment  or  calibration  by  the  user.  Our  experience  with  the  EyePhone  is  that  using  the  calculated  parameters 
gives  very  solid  stereo  perception  for  all  users  who  can  see  in  stereo,  with  no  need  for  tweaking  these  parameters.  Getting  this 
all  to  work  depends  on  getting  several  subordinate  things  right  -  the  specifications  for  the  optics  must  be  correct,  the  analysis 
of  the  optics  must  be  correct,  the  measurements  of  the  LCD  positions  and  dimensions  must  be  correct,  and  the  graphics 
software  for  setting  up  the  projections  must  be  correct. 

Before  these  calculations  were  done,  the  FOV  and  off<enter  parameters  had  been  tweaked,  through  a  long  process  of  uial  and 
error,  trying  to  get  the  image  to  look  right  The  most  effective  lest  was  to  try  to  get  the  image  of  a  5.75  centimeter  red  sphere 
to  be  the  right  size  and  stay  on  top  of  a  physical  5.75  centimeter  red  3-ball  (from  the  game  of  pool)  with  a  Polhemus  sensor 
inside  as  the  3-ball  was  moved  around.  This  3-bail,  which  has  two  pushbuttons  on  it,  is  one  of  our  manual  input  devices.  The 
FOVs  for  a  single  eye  which  were  finally  arrived  at  through  these  subjective  tests  were  a  horizontal  FOV  of  80  degrees  and  a 
vertical  FOV  of  60  degrees.  Since  the  EyePhone  does  not  allow  the  user  to  see  through  to  the  real  world,  the  subjective  tests 
had  to  be  done  by  repeatedly  raising  the  EyePhone  to  see  how  the  position  of  the  physical  3-balI  compared  with  the 
remembered  position  of  the  simulated  3-balI.  We  estimated  the  accuracy  to  be  about  2  degrees.  For  a  see-through  HMD, 
superimposing  a  3D  stereoscopic  image  onto  a  physical  object  of  the  same  known  shape  is  an  exU'emely  accurate  test, 
because  the  eye  can  simultaneously  compare  the  two  scenes  and  detect  liny  discrepancies.  The  acuity  of  the  human  eye  is 
considered  to  be  roughly  one  minute  of  arc. 

Figure  4.3  shows  that  the  horizontal  FOVs  for  the  two  eyes  in  the  EyePhone  partially  overlap.  The  region  of  overlap  (60.6 
degrees)  is  wide  enough  for  suong  stereoscopic  perception,  and  the  binocular  FOV  (90.0  degrees)  is  wide  enough  to  provide  a 
feeling  of  immersion  within  the  scene.  We  believe,  however,  that  a  wider  FOV  will  make  the  feeling  of  immersion  suonger. 
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90.0  degrees 


Figure  4.3  Binocular,  overlapped  and  single-eye  FOVs  for  the  EyePhone 


4.3  Correction  of  EyePhone  optical  distortion 

To  correct  the  image  on  the  screen  from  optical  distortion,  the  screen  image  must  be  predistorted  as  specified  by  the  function 
D'^.  This  would  radially  shift  all  the  pixels  in  the  image  by  some  amount.  This  has  not  yet  been  implemented  on  the  UNC 
HMD  system,  but  we  plan  to  use  a  pair  of  two-dimensional  tables  Tx[xv,yv]  and  TY[xv.yv]  in  this  implementation. 

We  have  a  512  x  512  pixel  screen  to  cover,  but  a  table  size  of  512  x  512  is  impractical.  We  expect  to  use  a  reduced  table  size, 
such  as  64  X  64,  and  interpolate  bilinearly  between  table  entries.  The  table  values  will  be  computed  off-line  using  the  formula 
forD'i. 

The  optics  map  object  points  of  height  rs  to  image  points  of  height  ry.  This  mapping  is  monotonic  and  so  its  inverse  exists. 
We  approximate  the  mapping  with  a  third-degree  polynomial  D  and  approximate  its  inverse  with  another  third-degree 
polynomial  D*^ 

*'sn  =  fvn  ^'sv  fvn^ 
in  which,  for  the  LEEP  optics,  ksv  is  -0.18. 

Figure  4.4  shows  the  graph  of  the  normalized  virtual  image  position  ryn  versus  the  normalized  screen  position  rgn  for  the 
LEEP  optics.  It  shows  that  the  third-degree  polynomial  approximation  of  the  distortion  function  D  is  quite  close  to  the  more 
accurate  graph  of  D  calculated  by  ray-tracing  through  the  optics.  The  second  graph  compares  the  third-degree  polynomial 
approximation  of  the  inverse  D'^  with  the  values  calculated  by  ray-tracing. 
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Figure  4.4  Graph  of  rvn  vs.  rsn  for  D  (kys  =  0.32)  and  ray-tracing  for  the  LEEP  optics; 
graph  of  rsn  vs.  rvn  for  D'^  (ksv  =  -0.18)  and  ray-tracing 


Figure  4.5  shows  a  grid  which  has  been  predistorted  by  the  function  D'^  When  viewed  through  the  LEEP  optics,  the  lines  of 
the  grid  appear  straight 


Figure  4.5  Grid  prcdistoncd  for  the  LEEP  optics 
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Although  the  prcdistortion  function  D'^  also  known  as  the  Virtual-Imagc-To-Scrcen  transformation,  has  not  yet  been 
implemented  on  the  UNC  HMD  system,  we  have  looked  through  the  LEE?  optics  at  the  predistorted  grid  printed  on  paper  to 
verify  that  the  grid  lines  do  appear  straight  The  predistorted  grid  in  Figure  4.5  is  full-sized  (79  mm  square)  and  may  te  viewed 
through  the  LEEP  optics.  The  lines  will  appear  straight  when  the  grid  object  is  in  close  contact  with  the  lens.  Note  that  in 
the  EyePhone,  the  LCD  screen  is  separated  by  a  gap  of  a  few  millimeters  from  the  LEEP  lens. 


4.4  Other  EyePhone  and  LEEP  parameters 

The  graph  of  Figure  4.6  shows  how  the  LEEP  optics  FOV  varies  with  the  eye  relief  distance  der>  the  distance  between  the  eye 
and  the  nearest  lens  surface.  With  the  nominal  eye  relief  of  29.4  mm,  the  FOV  for  the  LEEP  optics  is  •t4S  to  -45  degrees. 
(The  EyePhone’s  FOV  is  less  than  this  because  the  EyePhone’s  LCD  screen  does  not  fill  the  LEEP  object  field.)  If  the  eye 
was  able  to  get  closer,  the  FOV  would  increase,  but  an  eye  relief  of  25  to  30  mm  is  necessary  to  allow  people  with  spectacles 
to  use  the  system. 

When  moving  the  eye  closer  to  the  lens,  two  factors  contribute  to  an  increase  of  the  FOV  :  first,  any  point  on  the  virtual 
image  is  seen  over  a  larger  angle,  and  second,  more  of  the  LCD  screens  can  be  perceived.  Especially  for  the  EyePhone, 
moving  the  eye  closer  to  the  lens  does  increase  the  FOV  because  some  of  the  LCD  display  screen  is  vignetted  for  a  pupil 
distance  of  29.4  mm. 


Figure  4.6  Graph  of  FOV  vs.  dcr 


The  virtual  image  of  the  screen  is  formed  at  some  distance  zy  from  the  eye,  and  the  eye  must  accomodate  to  this  distance.  The 
graph  of  Figure  4.7  shows  that  Zy  is  very  sensitive  to  changes  in  the  distance  dob  object  plane  to  the  the  LEEP 
optics.  The  EyePhone  positions  the  LCD  screen  at  dob  =  16.4  mm  from  the  nearest  lens  surface  (measured  along  the  optical 
axis).  This  value  of  dob  results  in  an  image  distance  of  Zy  =  398.2  mm  and  a  magnification  of  mys  s  9.66 .  As  the 
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object  approaches  the  object  focal  point  of  the  lens  ( dob  =  ^0.7  mm)  the  image  distance  goes  to  infinity.  Howevcf,  such  a 
positioning  seems  undesirable  because  of  the  conflict  between  convergence  and  accomodation. 


LEEP  LEEP 

touches  focal 

object  plane  distance 


dob 

Figure  4.7  Graph  of  Zy  vs.  dob 


The  discussion  of  the  resolution  of  the  LCD  display  screens  in  the  EyePhone  is  complicated  by  two  competing  ways  of 
specifying  resolution  in  a  color  display  -  by  color  pixels  or  by  the  RGB  component  cells  of  the  pixels.  The  LCD  used  in  the 
EyePhone  has  a  monochrome  array  of  320  x  240  individually  controllable  light-producing  cells,  with  red,  green  and  blue 
niters  overlaid  to  divide  the  cells  into  three  equal-sized  groups.  A  triad  of  one  red,  one  green  and  one  blue  cell  makes  up  a 
color  pixel.  There  are  76,800  cells  (320  x  240),  and  therefore  25,600  color  pixels.  The  resolution  of  the  EyePhone  in  terms  of 
color  pixels  is  thus  approximately  184.7  x  138.6. 

Table  4.2  lists  all  the  parameters  for  the  EyePhone  with  respect  to  the  model  developed  in  this  paper.  All  the  angles  (0  and 
FOV  parameters)  were  calculated  from  the  other  parameters  as  described  above. 

The  LEEP  optics  are  used  in  other  HMDs  besides  the  EyePhone,  and  a  new  model  of  EyePhone  with  a  different  LCD  display 
is  also  being  developed.  Some  of  the  parameters  in  Table  4.2  depend  only  on  the  LEEP  optics  (  0axes>  (^axes  )•  Other 
parameters  with  specific  values  in  the  table  describe  the  specific  configuration  in  the  EyePhone  (Models  1  and  2)  of  the  LCD 
screen’s  size  and  position  in  front  of  the  LEEP  optics. 

The  rubber  diving  mask  of  the  EyePhone  holds  the  face  and  eyes  in  a  fairly  constant  position  with  respect  to  the  LEEP 
optics,  although  some  individuals  have  deeper-set  eyes  than  others.  Decreasing  d^r  by  moving  the  eyes  closer  to  the  LEEP 
lens  would  cause  more  of  the  object  field  to  be  seen  and  the  value  of  wj  which  was  chosen  to  be  the  highest  value  of  the  field 
would  then  have  to  be  increased  accordingly.  If  the  same  nominal  eye  relief  used  in  this  paper  (29.4  mm)  is  assumed,  then  the 
distortion  model  can  be  applied  to  an  HMD  using  the  LEEP  titles  and  a  different  screen  using  the  coefficients  calculated  (kvs 
a  0.32,  ksv  »  -0.18). 
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Parameter 

Symbol 

Value 

for  EyePhone 

maximum  object  field  radius 

ws 

28.1  mm 

maximum  virtual  imaee  field  radius 

Wy 

271.55mm 

transversal  magnification 

msv 

9.66 

eve  relief 

dec 

29.4  mm  (nominal) 

object  plane  distance  (LCD  screen  to  LEEP  lens  surface) 

dob 

16.4  mm 

distance  from  eye  to  virtual  image  plane 

2v 

398.2  mm 

coefficient  of  optical  distoition  for  D 

kvs 

0.32 

coefficient  of  optical  distortion  for  D‘^ 

l^sv 

-0.18 

angle  between  optical  axes 

daxes 

0  degrees 

distance  between  optical  axes  (at  front  surface  of  optics) 

daxes 

64  mm 

screen  center  offset  from  optical  axis 

(Cxs,Cys) 

screen  resolution 

ResH  X  Resv 

185  X  139  pixels 
(color  triads) 

inter-pupillary  distance  of  user 

IPD 

angular  position  of  virtual  image  of  right  edge  of  LCD 

CR 

45.0  degrees 

angular  position  of  virtual  image  of  left  edge  of  LCD 

0L 

30.3  degrees 

angular  position  of  virtual  image  of  top  edge  of  LCD 

0T 

31.8  degrees 

angular  position  of  virtual  image  of  bottom  edge  of  LCD 

0B 

26.6  degrees 

single  eye  vertical  field-of-view 

FOVy 

58.4  degrees 

single  eye  horizontal  field-of-view 

FOVh 

75.3  degrees 

overlapped  field-of-view 

60.6  degrees 

binocular  field-of-view 

90.0  degrees  . 

Table  4.2  Parameters  for  the  VPL  EycPhone,  models  1  and  2 


To  determine  the  model  parameters  for  other  HMDs  that  use  the  LEEP  optics,  the  distance  dob  from  the  LEEP  optics  to  the 
display  screen  must  first  be  known.  This  will  determine  the  distance  to  the  virtual  image  Zy  and  the  magnification  mgy.  An 
eye  relief  distance  d^r  must  also  be  measured  or  assumed.  This  will  determine  the  radius  of  the  object  and  image  fields  wg 
and  wy.  The  object  and  image  fields  describe  what  could  be  seen  if  the  object  field  was  completely  filled,  regardless  of  how 
completely  the  display  screen  does  fill  the  object  field.  To  compute  the  FOVs  for  the  virtual  image  of  the  display  screen,  the 
positions  of  the  edges  of  the  display  screen  (points  L,  R.  T  and  B)  must  be  measured  in  the  object  plane  with  the  optical  axis 
as  the  origin.  Cranking  these  measurements  through  the  model  will  give  the  angular  positions  of  the  edge  points  (  or, 

0T.  CB  ).  and  thus  the  horizontal  and  vertical  FOVs  (  FOVh,  FOVy  )  for  a  single  eye.  To  find  the  binocular  field-of-view 
FOVbin  for  both  eyes,  the  angles  between  the  optical  axes  daxes  must  be  taken  into  account  The  position  of  the  display 
screen’s  center  with  respect  to  the  optical  axis  must  also  be  measured  in  order  to  properly  set  up  the  perspective  projection. 


5.  SUMMARY  AND  CONCLUSIONS 

The  optics  model  presented  in  this  paper,  if  implemented  correctly,  will  generate  undistorted  orthostereoscopic  images  for  the 
user’s  two  eyes. 

To  calculate  the  display  parameters  needed  by  the  model  for  the  particular  HMD  being  used,  it  is  necessary  to  know  or 
measure  the  specifications  of  the  optics,  and  the  relative  positions  of  the  eyes,  optics,  display  screens,  and  head  position 
sensor.  The  construction  of  the  HMD  must  be  rigid  enough  that  these  values  will  not  vary  from  day  to  day.  If  these 
parameters  for  the  HMD  are  known,  then  several  important  derived  parameters  can  be  calculated  -  the  FOVs,  the  screen-center 
offset  for  the  perspective  projection,  the  angle  between  the  optical  axes,  and  the  coefficients  for  the  optical  field  distortion 
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function.  Calculating  the  values  of  these  parameters  is  much  more  accurate  than  attempting  to  measure  them  subjectively 
with  users. 
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